1. Field of the Invention
The present invention relates to a device for automatically operating a GUI (Graphical User Interface) for the purpose of usability evaluation or operational test of a system having a GUI or generation of an operation macro thereof and to an operation macro execution device for executing the operation macro of the system.
2. Description of Related Art
An evaluation method using a check list is disclosed as a method for evaluating an usability of a system operated by a user in the following articles:
Article (1): CHI '90 Conference Proceedings, ISBN-0-201-50932-6, pp. 249-256 (Apr. 1-5, 1990); PA1 Article (2): "Practical Evaluation Method of User Interfaces", ISBN-4-303-72860-8, pp21-27 (1993); and PA1 Article (3): The Technical Reports, the Information Processing Society, Vol. 93, No. 80, pp. 17-24 (Sep. 17, 1993)). PA1 Article (4): Japanese Patent Application laid-open No. H7-253865; and PA1 Article (5): Japanese Patent Application Laid-open No. H6-95780. PA1 Article (6): "Visual & Textual Consistency Checking Tools for Graphical User Interface", Technical report, CAR-TR-828, Human-Computer Interaction Laboratory, Center for Automation Research, University of Maryland (available from http://www.cs.umd. edu:80/projects/hcil/research/tech-report-list.html1#1996 through World Wide Web, May 1996). PA1 Article (7): Japanese Patent Application Laid-open No. H7-84829; PA1 Article (8): Japanese Patent Application Laid-open No. H7-84765; PA1 Article (9): Japanese Patent Application Laid-open No. H6-28209; and PA1 Article (10): Japanese Patent Application Laid-open No. H2-41529. PA1 Article (11): NIKKEI Open System, No. 40, pp. 247-257 (July, 1996). PA1 Article (12): Japanese Patent Application Laid-open No. H6-348481; PA1 Article (13): Japanese Patent Application Laid-open No. H6-274329; PA1 Article (14): Japanese Patent Application Laid-open No. H5-173741; PA1 Article (15): Japanese Patent Application Laid-open No. H5-35382; and PA1 Article (16): Japanese Patent Application Laid-open No. H1-237833. PA1 a program execution start unit for causing the program execution unit to start an execution of the execution program of the system; PA1 a screen image recording unit for recording an image on a display screen which is displayed by the program execution unit; PA1 an operation object table recording unit for producing an operation object table from data of operation objects detected by the operation object detection unit and recording the operation object; PA1 a object table stack managing unit for adding identifiers to a table stack of identifiers of the tables of operation objects or deleting identifiers from the table stack of identifiers of the tables of operation objects; PA1 an operation object selection unit for selecting data of the operation objects from the tables of operation objects recorded in the operation object recording unit; PA1 an operation event generation unit for generating an event for executing an operation on operation objects of the system; and PA1 a program execution end unit for ending the execution of the execution program of the system which is being executed by the program execution unit, PA1 wherein the operation object table recording unit extracts data which is not recorded in the operation object table recording unit from the data of the operation objects detected by the operation object detection unit, produces the operation object table by using the data and records the table, instructs the table stack managing unit to add identifiers of the newly recorded operation object table to the table stack, and, in a case where all of the data of the operation objects detected by the operation object detection unit have been recorded in the operation object table recording unit already, instructs the program execution unit to terminate the execution of the execution program of the system by means of the program execution unit and, in a case where, in the data of the operation objects detected by the operation object detection unit, there are data which are not recorded in the operation object table recording unit as yet, instructs the operation object selection unit to select data of operation objects from the tables of operation objects recorded in the operation object table recording unit, PA1 wherein the operation object selection unit searches the tables of operation objects having the top identifier in the table stack from the operation object table recording unit, searches data of operation objects which are not operated as yet from the searched operation object table and instructs the table stack managing unit to select one of data of operation objects which are not operated as yet, if any, or to select one of data of operation objects from the tables of operation objects recorded in the operation object table recording unit if there is no such data, or to delete the identifier in the top of the table stack until all identifiers in the table stack are deleted and then repeats the search processing of the data of the non-operated operation objects, PA1 wherein the operation event generator unit generates an event for executing an operation of operation objects of the system using the data of operation objects selected by the operation object selection unit and outputs it to the program execution unit, PA1 wherein the program execution end unit instructs the program execution start unit to start the execution of the execution program of the system after the execution of the execution program of the system by the program execution unit is ended, PA1 wherein the program execution start unit instructs the operation object selection unit to select data of operation object, after it instructs the program execution unit to start the execution of the execution program in response to the instruction from the program execution end unit, and PA1 wherein, when one of data of non operated operation objects is selected in response to the instruction from the program execution start unit instructing the selection of data of operation object, the operation object selection unit searches, from the operation object table recorded in the operation object table recording unit, data of operation object to be executed in order to shift a content on the display screen from an initial content of the system till a content displaying the selected data of operation object. PA1 a program macro recording unit for recording the operation macro; and PA1 an operation event unit for generating an event for executing an operation of the operation objects of the system, PA1 wherein data of operation object output from the operation object detection unit includes at least name of the operation object, label of the operation object, coordinates of a display area of the operation object and a label of a window on which the operation object is located, PA1 wherein the operation event generator unit refers the operation macro recorded in the operation macro recording unit, extracts data of operation object for executing the operation macro selected on the display screen by a user from the data of operation object detected by the operation object detection unit, generates an event for executing an operation of operation objects having the extracted data and outputs the event to the program execution unit.
In the user interface evaluation method disclosed in either one of these articles, an operator checks on a display screen whether or not check items listed on a check list are satisfied, for example, whether or not an information displayed on the screen can be understood easily, while changing the content of display by operating the system. Thus, the operator can find certain ones of the check items which do not satisfy definitions thereof, so that it is possible to improve the usability of the system by improving design of the user interface for the certain items.
Further, an automatic operation device of a system operated by a user is disclosed in the following articles:
In the device for automatically executing a system operation disclosed in the Article (4) or (5), an operator preliminarily registers and stores operation procedures and the registered operation procedures can be automatically executed sequentially without operation of the operator by reading out the procedures sequentially and sending instructions for executing the individual procedures to the system sequentially.
Further, a device for evaluating an usability of a system having the GUI is disclosed in the following article:
In the GUI evaluation device disclosed in the Article (6), data of font and size of label letters on buttons which are one of GUI operation objects are extracted from a source program of a GUI portion of a system and output in a table format. From the table, an evaluator can compare font and size of label letters on the buttons throughout the system and, if letters, the font and size of which are different from others, are used on some buttons, he can know that the consistency of letter is not maintained throughout the system.
Further, a device for testing an operation normality of a system operated by a user is disclosed in the following articles:
Further, as means for testing the normality of operation of a system having a GUI, there are software products WinRunner available from Mercury Interactive, USA, SQA Robot available from SQA, USA, SatellitePackage available from Performance Awareness, USA and VisualTest available from Microsoft, USA. GUI test device using these softwares is introduced in the following article:
In a GUI test device which uses the commercially available software product introduced in the article (11), in order to test an operation of a system responsive to an input operation of a user, a script for automating the input operation of the user is described. Alternatively, it is possible to produce the script by recording the input operation. It is possible to detect an abnormal operation of a system by automatically executing the user input operation by using such script and recording a result of system operation. The user interface test device disclosed in the Article (8) or (10) makes possible to test the system operation by producing a script by recording a user input operation, reproducing the input operation by using the thus produced script and automating the input operation, similarly to the GUI test device using the commercially available software product introduced in the Article (11). Further, in order to test a system operation in response to a user input operation, the user interface test device disclosed in the Article (7) describes a script for automating a user input operation, similarly to the GUI test device using the commercially available software product introduced in the Article (11). By reproducing the user input operation using the script to automate the user input operation, it is possible to test the system operation. On the other hand, in the user interface test device disclosed in the Article (9), it is possible to test a system operation by operating the system by not a user but a robot and recording a result of the system operation. In this case, however, it is necessary to produce a program for controlling the robot.
The following articles disclose a device for producing an operation macro of a system to be operated by a user:
In the operation macro producing device disclosed in each of the Articles (12) to (14), a user inputs a series of procedures for a system sequentially and the operation macro is produced by recording the procedures. On the other hand, in the operation macro producing device disclosed in the Article (15) or (16), procedures performed by a user are monitored during a time for which the user uses a system and a series of procedures, the frequency of use of which exceeds predetermined times, are automatically registered as an operation macro. As mentioned, in the operation macro producing device disclosed in each of the Articles (12) to (16), the series of procedures which are utilized by the user frequently can be executed by the operation macro by producing the operation macro from the procedures performed by the user, so that the operation efficiency is improved.
However, in the user interface evaluation method disclosed in any of the Articles (1) to (3), in which it is judged by the evaluator whether or not the content of the display screen of the system follows the check list items, it is necessary for the user to shift the content on the screen of the system by operating the system sequentially. Therefore, when the size of the system becomes large, the number of procedures to be executed in order to comprehensively shift the content of the system screen is increased, which is very troublesome. Further, when the system size and hence the complication of structure of operation procedures become large, the risk that a user forgets execution of some procedures and a comprehensive confirmation of all contents of the display screen of the system becomes impossible is increased correspondingly.
Tn the system operation automatic execution device disclosed in the Article (4) or (5), it is possible to execute the operation of the system not manually but automatically. Therefore, by executing the user interface evaluation method disclosed in any of the Articles (1) to (3) by utilizing such device, a system operation of an evaluator becomes unnecessary. However, in the system operation automatic execution device disclosed in the Article (4) or (5), it is necessary for the user to preliminarily describe and store the procedures of the system operation to be automatically executed. Therefore, when the system size becomes large, the number of procedures to be executed to comprehensively shift the content of the system screen and, in order to preliminarily describe the operation procedures, enormous labor and time are necessary. Further, the risk that the user forgets to describe a portion of the procedures is increased correspondingly to the increase of the system size and hence the structure of operation procedures and the comprehensive shift of the contents of the display screen of the system becomes impossible.
Further, in the GUI evaluation device disclosed in the Article (6), in order to obtain data of font and size of the label letters on the buttons, a source program of a GUI portion of the system is necessary. Therefore, in a case where only the execution program of the system can be utilized, it is impossible to use the GUI evaluation device disclosed in the Article (6).
In the user interface test device disclosed in any of the Articles (7), (8) and (10) and the GUI test device disclosed in the Article (11) and commercially available as the software product, the script for automating the user input operation is necessary and the tester must produce the script by recording the input procedures of the user or through manually description. Therefore, the larger the system size provides the larger the size of script necessary for testing the system operation comprehensively and thus, in order to preliminarily record or describe the operation procedures, enormous labor and time are necessary. Further, the risk that the user forgets to describe a portion of the procedures is increased correspondingly to the increase of the system size and hence the structure of operation procedures and the comprehensive shift of the contents of the display screen of the system becomes impossible. Further, in the user interface test device disclosed in the Article (9), it is necessary to produce an operation control program for controlling the operation of the robot for operating the system. Therefore, when the size of the system becomes large, the size of the robot operation control program necessary to comprehensively test the system operation becomes enormous and the production of such program requires considerable time and labor. Further, when the system size becomes large and hence the complication of structure of operation procedures is increased, the risk that a user forgets to describe a program portion for operating some procedures is high and it becomes impossible to comprehensively test the system operation.
Further, in the operation macro producing device disclosed in any of the Articles (12) to (16), in order to produce the operation macro, the user must operate the system. Therefore, it is impossible to produce the operation macro before the user operates the system and to supply the operation macro to the user.